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Pre-Service Teachers' Impact on Student Learning: Three- Year Study of the 
Impact on Student Learning Project 
Abstract 

With the demand for a demonstration of continuous progress as an accountability 
gauge in public schools, teachers are compelled to examine assessment data for each of 
their pupils \ This data analysis, in turn, should help teachers gauge their instructional 
practices and differentiate instruction so that all can reach proficient levels of 
achievement. Teacher education programs, too, have been asked to provide performance- 
based programs that demonstrate pre-service teachers can positively affect individual 
student learning. One way for teacher education programs to demonstrate this measure of 
success is by having pre-service teachers show their understanding of differentiated 
instmction, test data collection and analysis, and reflection. To reach this goal our 
Teacher Education faculty implemented an assignment titled “Impact on Student 
Learning Project” during the Fall 2006 semester. The intent of this paper is to describe 
the project and summarize the results culled from pre-service teachers who have 
completed the project from Fall 2006 until Spring 2009. 
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With the demand for a demonstration of eontinuous progress as an accountability 

gauge in public schools, teachers are compelled to examine assessment data for each of 
their pupils . This data analysis, in turn, should help teachers gauge their instructional 
practices and differentiate instruction so that all can reach proficient levels of 
achievement. Teacher education programs, too, have been asked to provide performance- 
based programs that demonstrate pre-service teachers can positively affect individual 
student learning. One way for teacher education programs to demonstrate this measure of 
success is by having pre-service teachers show their understanding of differentiated 
instruction, test data collection and analysis, and reflection. To reach this goal our 
Teacher Education faculty implemented an assignment titled “Impact on Student 
Learning Project” during the Fall 2006 semester. The intent of this paper is to describe 
the project and summarize the results culled from pre-service teachers who have 
completed the project from Fall 2006 until Spring 2009. 

Theoretical Framework 

Every country has mandated standardized testing to monitor issues of 
accountability and to rank its education program with those throughout the world. 
Governments and school administrators are responsible for the dissemination and 
analysis of test results to their constituents. These standardized tests are an important 
gauge to measure learning. However, most teachers judge achievement through the 
instruction and assessments that take place in their individual classrooms. Designing and 
interpreting formative and summative assessments is a critical step in gauging the 



^ The term student and students refer to the pre-service teachers, and the term pupU and 
pupils refer to the children in their classroom 
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students’ achievement of the school’s curriculum in relationship to mastery of the 
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objectives of standardized tests. 

Teacher educators are looking for ways to demonstrate pre-service teachers’ impact 
on their pupils’ learning. Hamel and Merz (2005) state that teacher evaluation has 
typically been “grounded in the behaviors, qualities, and growth that teachers exhibit 
rather than in changes exhibited by students. . .’’ (p. 158). The paradigm shift— to evaluate 
teachers by their pupils’ achievement— changes the emphasis from teaching to learning. 

Emphasizing learning instead of teaching brings a different set of skills and 
knowledge to monitor pupil achievement. First, pre-service teachers must be able to 
understand how pupils think from various perspectives (Hamel & Marz, 2005). Because 
of the multiple dimensions and complexity of the classroom (Hargreaves, 1994; Lortie, 
1975), pre-service teachers tend to teach to the “whole,” teach to the “middle,” covering 
the content without regard to individual learning styles and diverse capabilities of pupils. 
Hamel and Merz (2005) explain that pupil understanding is elusive, influenced by 
multiple factors; therefore, schools and teachers deliver the same content to “all sections 
of a class or use common textbook or program based assessments” (p. 164). 

The most complex dimension of teacher knowledge is understanding how and why 
pupils respond to instruction (Hamel & Merz, 2005). This approach emphasizes the 
ability to interpret pupil responses within the contexts of the school and classroom 
settings and the diverse pupil population. Hamel and Merz (2005) explain novices must 
learn how their own “understanding of student understanding” can drive instruction: 

“. . .novice teacher learning is often reflected less in ‘what happens’ during a particular 
lesson or unit (whether the sequence worked as intended, whether pupils exhibited 
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learning goals) but more in how the pre-service teacher makes sense of student response” 



(p. 159). 

Recent models of teaching and learning include formative assessment as an important 
element of instruction (Shepard et ah, 2005). Having pre-service teachers engage in 
ongoing assessment keeps them attuned to differentiating instruction to meet the needs of 
all pupils, as well as to continually reflect on their lesson design and instructional 
strategies. Otero (2006) writes that through formative assessment, “the teacher learns 
about students’ conceptual understandings at given points in time and, therefore, becomes 
better equipped to help students move further in their understandings with respect to an 
academic objective” (p. 250). 

Hamel and Merz (2005) write that teacher reflection, flexibility in thinking, multiple 
interpretations, making adjustments, and articulating existing frames of reference are the 
keystones for teacher competency and growth. Reflection encourages novices to think 
and question, “why they teach, what they teach, and to whom they teach” so that 
instructional strategies can be effective for all pupils (Pultorak & Barnes, 2009). 
Researchers have found that novices have difficulty gauging the strengths and 
weaknesses of a lesson. Risko, Vukelich, and Roskos (2009), having researched teacher 
reflection the last two decades, observed a shift from “hands-off instruction that had 
limited impact on developing teachers’ reflective abilities to forms of guided instruction 
that deepened reflective thinking” (p. 48). Researchers have employed different forms of 
instruction to assist novices to reflect systematically on their instruction. Some of these 
methods include guided questions, case analysis, journal writing, small group discussion, 
and peer coaching (Risko, Vukelich & Roskos, 2009). 
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Impact on Student Learning Project 

Our Teacher Education faculty believes formative assessment and data analysis are 
also critical to meaningful reflection. Thus, we implemented the Impact on Student 
Learning Project to have our pre-service teachers employ various instructional methods, 
reflect on the success of the methods vis a vis pupil achievement, and implement 
different strategies as needed. 

Contextual and Program Background 

Our Teacher Education Program, located in a small college of a major university in 
Pennsylvania, is both state and nationally accredited. We offer initial programs in early 
childhood and elementary teacher certification. Eligible students enroll in the program 
their junior year (fifth semester), and they are placed in cohorts throughout the program. 
Students complete a field assignment each semester in both urban and suburban settings, 
gradually increasing the number of days and hours in the schools as they progress 
through the program. Eull-time faculty members who instruct the students during a 
particular semester supervise during the field observations, and each student is assigned 
to a specific host teacher in each school. Currently, 80 students are enrolled in the 
program, and 20 faculty members teach in this program, as well as in the Master’s in 
Education in Teaching and Curriculum program. 

Students are first assigned a variation of the Impact on Student Learning Project 
during their seventh semester (the semester before they student teach). Seventh-semester 
students are placed in an urban setting 12 hours a week for 9 weeks. During this period, 
students plan a project that impacts the specific needs of the children in that classroom. 
The host teacher in consultation with the college supervisor may help the pre-service 
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teacher select the type / scope of the project from a suggested list of projects that are 
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included in the syllabus. 

The Project 

The objectives for our pre-service teachers to become more proficient are these: (1) 
determine the effect of instruction on all pupils’ learning; (2) guide decisions about future 
instruction and plans to improve upon every pupil’s performance; (3) communicate 
performance results to others; and (4) reflect on their performance as teachers. 

Early in the teacher certification preparation program, the pre-service teachers receive 
instmction in differentiating their methods to meet the needs of their pupils. Much of this 
background comes from the work of Carol Tomlinson. She states, “Learning to 
differentiate well requires rethinking one’ s classroom practice and results from an 
ongoing process of trial, reflection, and adjustment in the classroom itself’ (Tomlinson & 
Imbeau, 2010, p. 13). The pre-service teachers are prepared in the following critical 
components of lesson planning to meet the needs of all pupils: 

1 . Designing instruction appropriate to their pupils’ stages of development, learning 

styles, strengths, and needs. 

2. Selecting approaches that provide opportunities for different performance modes. 

3. Adjusting instruction to accommodate pupils’ learning differences or needs. 

4. Being aware of different cultural contexts and aligning learning opportunities to 

address these differences. (Tomlinson & Imbeau, 2010, p. 5) 

Using “backward design” (Wiggins & McTighe 2006), the pre-service teachers learn 
that differentiated instruction can only be implemented after they clearly articulate what 
the pupils must learn through measurable objectives that align to the curriculum and 
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standards. The next step is to devise assessments to measure if the objectives have been 



met. Then, by paying careful attention to the assessment goals, they are ready to plan 
instruction that meets the needs of all their pupils. By planning “backwards,” the pre- 
service teachers reach an appreciation for the need to use different methods of instruction 
to have all pupils reach the objectives. 

Reflection is a major tenet of our Teacher Education Program. Each semester pre- 
service teachers are obligated to systematically reflect on their learning. As they progress 
through the program and participate in field experiences, they reflect on their 
understanding of what they learned theoretically in their college classes to the realities of 
practice in the elementary school classroom. Einally, they are introduced to ways to 
compile pupil data to analyze the success or failure of their instruction in light of test 
results. Through reflecting on their analysis, they are able to construct better lessons and 
differentiate instructional strategies to meet the needs of all pupils. 

The Project is introduced to our pre-service teachers during the semester of their final 
field experience. Our aim at this stage is for pre-service teachers to understand all the 
nuances of the project so they can complete a more detailed, larger-scale plan when they 
student teach. Pre-service teachers receive specific instruction in the required components 
of the project prior to their field experience and have several opportunities for peer and 
instructor review throughout the semester. 

The assignment is structured so that pre-service teachers feel confident in this first 
attempt to understand and measure their impact on pupil learning. As previously 
explained, the host teacher and college supervisor help the student develop a project to 



implement and assess. 
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They select an individual pupil, a group of pupils, or an entire class whom they are 

teaching and/or tutoring to evaluate the impact of their instruction on pupil learning. 
During the field experience, our pre-service teachers are involved in tutoring. When they 
student teach, they have the opportunity to assess on a larger scale; for example, an entire 
class of pupils. During student teaching, they select a series of at least three lessons or an 
entire unit and decide on a method of data collection to measure the impact of their 
instruction upon pupil learning, using assessments that will generate data suitable for 
analysis. These assessments include inventories, pre- and post-tests, teacher-made 
quizzes and tests, to name a few. 

Major Components 

The six major components of the project will be described, and Emily’s work will be 
included to demonstrate the structure and details that are expected. 

Contextual Factors 

In assessing the impact of instruction on pupil learning, pre-service teachers interpret 
the results within the contexts of the school and classroom settings and the diverse pupil 
population. Knowledge of contextual factors is important because these factors often 
explain pupil behaviors and achievements. In their analysis, pre-service teachers 
investigate these contextual factors for those pupils they are evaluating: 

■ geographic location, community and school population, socio-economic profile, 
and race/ethnicity 

■ physical features of setting, availability of equipment/technology and other 
resources 



3 All names are pseudonyms. 
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■ pupil characteristics such as age, gender, race/ethnicity, exceptionalities 

(disability and giftedness), developmental levels, culture, language, interests, 

learning styles or skill levels. 

As an example, Emily described Rose Elementary School, one of seventeen 
schools in the City School District, housing roughly 400 pupils: 

This small city pre-kindergarten through sixth grade school is located in the 
state’s capital, where the population consists of nearly 47,000 residents and 
24,000 housing units (U.S. Census Bureau). Although the majority of the pupil 
population is African American, there are also many other races and ethnicities 
represented in the school. The school houses 298 African American pupils, 5 1 
Hispanic pupils, 44 Caucasian pupils, 4 Asian pupils, and 1 American Indian 
pupil. Additionally, Rose Elementary is located in an urban portion of the city, so 
the socio-economic status of the population is low. Because of this, roughly 390 
pupils, or 98%, are eligible for free or reduced lunch provided by the school. 

At Rose Elementary, I was placed into Ms. Smith’s second grade classroom 
for my first rotation. In her classroom, there were seven boys, ten girls, and a 
wide range of learning styles and abilities. Moreover, this classroom contained a 
wide variety of behavior problems. Most of the students in the class were 
diagnosed with Oppositional Defiant Disorder (ODD), with typical behaviors 
including refusal to complete tasks or follow directions, being spiteful, or 
destruction of property. Additionally, my mentor teacher believed that a few of 
the students suffered from Mental Retardation (MR) and Conduct Disorder; 



however, these were merely her professional opinions. 
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In most models of instructional planning, teachers must make decisions about 
assessment before instruction begins. With this understanding of the importance of 
writing, implementing, and assessing measurable objectives, our pre-service teachers, 
with the assistance of their host teachers and College supervisors, select two to four 
measurable objectives appropriate for their pupils. In writing the objectives, students are 
to address the following questions: 

• What will pupils accomplish during this lesson? 

• To what specific level (i.e. 75% accuracy) will the pupils perform a given task in 
order for the lesson to be considered satisfactorily accomplished? 

• Exactly how will the pupils show that they understood and learned the goals of 
your lesson? 

• How do the objectives align with the state and/or national standards? 

Finally, they communicate the objectives to their pupils. 

As an example, Emily, an elementary education pre-service teacher, completed 
her project with Jake, who was currently reading below grade level. Emily explained: 

My host teacher told me that she had an ideal student for the assignment, Jake 
Miller . Once I knew the student with whom I would be working, I spent the next 
week observing Jake, so I could learn about his interests, abilities, and 
weaknesses in the classroom. Jake knows all short vowels and long vowels 
sounds, and has kindergarten and first grade sight words mastered. His 
comprehension skills are good, and he has the ability to answer basic facts and 
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higher-level questions when asked about what he has heard or read. Additionally, 

he has excellent vocabulary and processing skills. 

Emily noted three measurable objectives: 

1. Objectives for first activity: 

Given flashcards with words beginning with th-, eh-, sh-, and wh-, the pupil 
will organize them into categories according to their initial sound with an 
acceptable level of accuracy as described on a rubric. 

2. Objectives for second activity: 

Given flashcards depicting various images of diagraphs, the pupil will 
recognize th-, eh-, sh-, and wh- beginning sound words with an acceptable 
level of accuracy as described on a rubric. 

3. Objectives for third activity: 

a. Given flashcards depicting various images of diagraphs, the pupil will 
recognize th-, eh-, sh-, and wh- beginning sound words with an acceptable 
level of accuracy as described on a rubric. 

b. Given flashcards with spelling words, the pupil will correctly spell the word 
with an acceptable level of accuracy as described on a rubric. 



Assessment Plan 

Pre-service teachers match their assessments, both formative and summative, with 
the objectives. They describe the assessments they will conduct before, during, and after 
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instruction. They may assess pupils in an authentic/alternative manner, a traditional 
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manner, or a combination of both. 

As an example, Emily explained how she first assessed Jake using a spelling 
inventory. Because of his scores, Jake was placed in the “Letter Name- Alphabetic” 
spelling stage (Bear, Invernizzi, Templeton, & Johnston, 2008). In order for Jake to 
move to the next stage, she designed three learning activities that helped improve his 
ability to recognize and spell sound diagraphs. These activities included a word sorting 
activity, a bingo game, and a creative review game. Emily observed Jake informally to 
assess his abilities, weaknesses, and improvements during each lesson with the use of 
specifically designed rubrics. Post-test results demonstrated Jake had improved, bringing 
his spelling stage to “Within Word Pattern.” Emily’s spelling assessment rubric she 
devised is presented below: 



Activity #1 ; Word Sort 


Criteria: 


Exemplary 
(10-8 points) 


Acceptable 
(8-5 points) 


Unacceptable 
(5-0 points) 


Given flashcards with 
words beginning with 
th-, ch-, sh-, and wh-, 
pupil will organize 
them into categories 
according to their initial 
sound with an 
acceptable level of 
accuracy. 


* Pupil completes the 
word sorting activity 
with 90-100% accuracy. 

* No teacher assistance 
is needed. 


* Pupil completes the 
word sorting activity 
with 76-89% accuracy. 

* Little teacher 
assistance is needed. 


* Pupil completes the 
words sorting activity 
with less than 75% 
accuracy. 

* Moderate teacher 
assistance is needed. 



Activity #2: Initial Sounds Bingo 


Criteria: 


Exemplary 
(10-8 points) 


Acceptable 
(8-5 points) 


Unacceptable 
(5-0 points) 


Given flashcards 
depicting various 
images of diagraphs, 
pupil will recognize th-, 
ch-, sh-, and wh- 
beginning sound words 
with an acceptable level 
of accuracy. 


* Pupil recognizes at 
least 3 th-, ch-, sh-, and 
wh- beginning sound 
words with their 
flashcard representation. 

* No teacher assistance 
needed. 


* Pupil recognizes 2 th-, 
ch-, sh-, and wh- 
beginning sound words 
with their flash card 
representation. 

* Little teacher direction 
needed. 


* Pupil recognizes only 1 
th-, ch-, sh-, and wh- 
beginning sound words 
with their flash card 
representation. 

* Moderate teacher 
direction needed. 
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Activity #3: Review Game 


Criteria: 


Exemplary 
(5 points) 


Acceptable 
(4-3 points) 


Unacceptable 
(2-0 points) 


Given flashcards 
depicting various 
images of diagraphs, 
student will recognize 
th-, ch-, sh-, and wh- 
beginning sound words 
with an acceptable level 
of accuracy. 


*Pupil recognizes at 
least 3 th-, ch-, sh-, and 
wh- beginning sound 
words with their 
flashcard representation. 


* Pupil recognizes at 
least 2 th-, ch-, sh-, and 
wh- beginning sound 
words with their 
flashcard representation. 


* Pupil recognizes only 1 
th-, ch-, sh-, and wh- 
beginning sound word 
with their flashcard 
representation. 


Given flashcards with 
spelling words, pupil 
will spell the word with 
an acceptable level of 
accuracy. 


* Pupil spells the correct 
word at least 90% of the 
time. 

* The pupil provides an 
excellent meaningful 
sentence. 


* Pupil spells the correct 
word 76-89% of the 
time. 

* The pupil provides a 
somewhat meaningful 
sentence. 


* Pupil spells the correct 
word less than *75% of 
the time. 

No sentence is given. 



Design for Instruction 

After administering the pre-assessment, pre-service teachers analyze pupil 
performance relative to the objectives. The results of the pre-assessment must be 
depicted in a format that allows the pre-service teachers to find patterns of pupil 
performance relative to each learning goal. By using a table, graph, or chart, pre-service 
teachers describe the emerged pattern to guide future instruction or modify the original 
objectives. 

Pre-service teachers describe at least three teaching activities that reflect a variety of 
instructional strategies/techniques and explain why they are planning those specific 
activities. In their explanation for each activity, pre-service teachers include the 
following: how the content relates to their objectives; how the activity stems from their 
pre-assessment information and contextual factors; what materials/technology they need 
to implement the activity; and how they plan to assess learning during and/or following 
the activity. 

Using Emily’s project as an example, she explained she began by researching ideas 



for learning activities that would help Jake after observing and conducting the pre- 
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assessment. She noted the textbook, Words Their Way, was a fantastic resource that 
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provided many creative instructional ideas. Most importantly, they all were designed for 
pupils who were in the middle of the Letter Name- Alphabetic spelling stage.) Emily’s 5- 
Day Instructional Plan is presented below: 



Day 


Planned Instruction 


Activity Objective(s) 


Activity 

Standard(s) 


L‘ Day 
9/29/09 


• Conduct a pre-assessment, which will 
be a spelling inventory with Jake. 
Observe abilities, strengths, and 
weaknesses throughout the assessment. 

• Score spelling inventory. 


Not Applicable 


Not Applicable 


Day 

10/8/09 


• Introduce first learning activity, word 
sort. This activity will reinforce 
pupil’s ability to say and recognize 
short vowels with diagraphs, 
specifically th-, ch-, sh-, and wh- 
beginning blends. 

• Have pupil recognize and pronounce 
the th-, ch-, sh-, and wh- beginning 
blends. Then have pupil recognize and 
read various th-, ch-, sh-, and wh- 
beginning words, making sure to assist 
as needed. Pupil will organize the 
words into categories according to 
their initial sound. 

• Activity duration 15 minutes. 


Given flashcards with 
words beginning with 
th-, ch-, sh-, and wh-, 
pupil will organize 
them into categories 
according to their 
initial sound with an 
acceptable level of 
accuracy as described 
on rubric. 


Standard I.IC: Use 
knowledge of 
phonics, word 
analysis (e.g., root 
words, prefixes, and 
suffixes), 
syllabication, 
picture and context 
clues to decode and 
understand new 
words during 
reading. 


3'‘‘* Day 

10/14/09 


• Introduce second learning activity 
“Initial Sounds Bingo.” Activity will 
reinforce the pupil’s ability to identify 
th-, ch-, sh-, and wh- beginning sound 
words. 

• Present pupil with flashcards depicting 
images of beginning diagraph words. 
Pupil will recognize and identify each 
flashcard with the “th”, “ch”, “sh”, or 
“wh” blend by placing a chip on the 
correct space on the bingo board. 
When pupil has four bingo chips in a 
row, game is complete. 

• Activity duration 15 minutes. 


Given flashcards 
depicting various 
images of diagraphs, 
pupil will recognize th- 
, ch-, sh-, and wh- 
beginning sound words 
with an acceptable 
level of accuracy as 
described on rubric. 


Standard I.IC: Use 
knowledge of 
phonics, word 
analysis (e.g., root 
words, prefixes, and 
suffixes), 
syllabication, 
picture and context 
clues to decode and 
understand new 
words during 
reading. 


4'” Day 
10/20/09 


• Introduce third learning activity, a 
review game. Activity will reinforce 


Given flashcards 
depicting various 


Standard I.IC: Use 
knowledge of 
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the concepts of past two learning 
activities by having pupil recognize 
and spell various th-, ch-, sh-, wh- 
beginning sounds words. 

• Pupil will participate in a review game. 
When pupil lands on a blue space on 
the game board, he will be asked to 
spell a word and then use that word in 
a meaningful sentence. When pupil 
lands on a green space, he will be 
asked to identify the correct initial 
sound depicted in the image. When 
pupil reaches the end of the game 
board, the review game is complete. 


images of diagraphs, 
pupil will recognize th- 
, ch-, sh-, and wh- 
beginning sound words 
with an acceptable 
level of accuracy as 
described on a rubric. 

Given flashcards with 
spelling words, pupil 
will correctly spell the 
word with an 
acceptable level of 
accuracy as described 
on a rubric. 


phonics, word 
analysis (e.g., root 
words, prefixes, and 
suffixes), 
syllabication, 
picture and context 
clues to decode and 
understand new 
words during 
reading. 

Standard LIE: 
Acquire a reading 
vocabulary by 
identifying and 
correctly using 
words (e.g., 
antonyms, 
synonyms, 
categories or 
words). 


5* Day 
10/21/09 


• Conduct a post-assessment, which will 
be a spelling inventory, to observe his 
abilities, strengths, and weaknesses. 

• Score the spelling inventory, looking 
for gains. 


Not Applicable 


Not Applicable 



Analysis of Student Learning 

Pre-service teachers conduct a summative assessment, which is correlated with the 
pre-assessment. They then summarize the data with descriptive statistics (e.g. gain 
scores, class means pre and post, composite records of observations) in the form of a 
graph, chart, or table. Finally, in a narrative, they interpret the data for evidence of impact 
on learning. 

Continuing with Emily’s example, she explained that Jake spelled only 5 out of 
26 words correctly in the pre-assessment. She then explained each activity and the gains 
that Jake made. She also noted that she took advantage of the activities he really enjoyed 



by making them more difficult. For example, he really enjoyed bingo. Thus, Emily 
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challenged him to play “bingo blackout,” meaning that he had to fill the entire bingo 



17 



board with chips. Emily explained these activities helped Jake improve substantially, 
noting he mastered diagraphs, as well as various sound blends. Emily’s Graph of Jake’s 
Progress is presented below: 



Words 

Spelled 

Correctly 

Pre-Assessment 

Post-Assessment 

8 

u M 


Feature 

Points 

Pre-Assessment 
^ Post-Assessment 

39 

27 ^ 

u E 







Total 

Number of 
Points 

^ Pre-Assessment 

^ Post-Assessment 
47 
32 

_u ■_ 



Reflection and Self-Evaluation 

In the final section of the project, pre-service teachers reflect on their performance 
and link their performance to pupil learning results, using the rubric from the Project as a 
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guide for reflection. They evaluate their performance and identify future action for 
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improved practice and professional growth. Emily reflected: 

It is without a doubt that I consider this instructional unit with Jake to be a major 
success, which the data support. Not only did Jake’s score on the spelling 
inventory drastically improve, he was placed into the next spelling stage, thus 
having his spelling and reading abilities flourish. 

At the start of the unit, Jake struggled a little when spelling, thus placing 
him in the Letter Name- Alphabetic spelling stage. His major difficulty was with 
beginning sound diagraphs, which could be seen in the pre-assessment. However, 
once Jake participated in the three learning activities, he was able to gain the 
necessary skills needed in order to improve. When the instructional unit was 
concluded, Jake scored in the Within Word Pattern spelling stage, and he was 
better able to identify, recognize, pronounce, and spell beginning sound diagraph 
words. 

I believe that the creativity of my learning activities was the strong point 
of this instructional unit, thus helping my impact be a success. However, nothing 
is ever perfect, and there is always room for improvement. With that being said, I 
would most definitely make some minor changes in the unit if I were to use it 
again in the future. I would have some of the activities focus on the letter blends 
found in the early stage of Within Word Pattern. By doing this, the unit would 
highlight both diagraphs and blends, and engage the pupil more thoroughly, 
essentially improving the instructional unit and the learning of the pupil based on 
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assessment results. By emphasizing both of these spelling objectives, it would 

hopefully bring even more success to the unit. 

The Results From Three Years of Data 
This section explains how we analyzed the results for the first three years of 
implementing the project. It also shows how we used the data collected from the rubric 
and students’ reflections to change the way we introduce the students to the project and 
assist them throughout their teacher preparation program. 

With input from cooperating teachers, college supervisors evaluate the projects. 
Since the assignment lasts anywhere from several days to several weeks, supervisors rely 
on the written analysis the pre-service teachers complete. Although all six categories of 
the rubric are graded during the field experience (see “Major Components” previously 
discussed), only three are judged on the project completed during student teaching: 
design for instruction, analysis of pupil learning, and reflection and self-evaluation. 

These three categories are evaluated as part of the e-portfolio, which all pre-service 
teachers complete. Therefore, the results discussed in this section will focus on the three 
components listed below. 



Rubric for Criteria 4, 5, and 6 



Criterion 


Unsatisfactory 


Satisfactory 


Above Average 


#4-Design for 

Instruction 

and 

Assessment 


• Does not incorporate 
community and 
classroom factors or 
diverse pupils’ 
characteristics; 

• fails to align with 
objectives; 

• relies on one 
approach; 

• little or no variety of 
formal/informal 
assessments; 

• not adapted to 


• Incorporates 
community and 
classroom factors and 
diverse pupils’ 
characteristics; 

• aligns with 
objectives; 

• uses variety of 
approaches; 

• some variety of 
formal/informal 
assessments conducted 
before, during and after 


• Incorporates a wide 
variety of community 
and classroom factors 
and diverse pupils’ 
characteristics; 

• aligns perfectly with 
objectives; 

• uses wide variety of 
approaches; 

• good variety of 
formal/informal 
assessments conducted 
before, during and after 
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learning needs of 
diverse pupils; 

• assessment measures 
not designed to assess 
progress in learning 


instruction; 

• adapted to needs of 
diverse pupils; 

• assessment measures 
somewhat designed to 
assess progress in 
learning 


instruction; 

• adapted to needs of 
divers pupils; 

• assessment measures 
effectively designed to 
assess progress in 
learning 


#5 -Analysis 
of Student 
Learning 


• Data not summarized 
in graphs or tables; 

• interpretation fails to 
include evidence of 
impact on learning 


• Data summarized in 
table or graph format; 

• data correctly 
analyzed and 
interpreted for evidence 
of impact on learning 


• Data accurately 
summarized in table or 
graph format with 
descriptive statistics; 

• data correctly and 
meaningfully analyzed 
and interpreted for 
evidence of impact on 
learning 


#6 -Reflection 
on Student 
Learning 


• Discusses implications 
of results for instruction 
for class as a whole; 

• identifies few or no 
changes in teacher 
actions; 

• does not evaluate 
teaching strengths and 
weaknesses 


• Discusses 
implications of results 
for instruction for 
individual pupils; 

• identifies actions 
needed to improve pupil 
learning based on 
assessment results; 

• evaluates teaching 
strengths and 
weaknesses 


• Meaningfully 
discusses implications of 
results for instruction for 
individual pupils at 
different levels; 

• identifies and explains 
further actions to 
improve pupil learning 
based on assessment 
results; 

• evaluates teaching 
strengths and 
weaknesses and 
identifies areas for 
professional growth 



The rubric contains three levels of achievement for each category: unacceptable, 
acceptable, and exemplary. These gradations were assigned the following points in order 
to align with our accountability management system to which our College subscribes: 1, 
unsatisfactory; 2, satisfactory; and 3, above average. 

We implemented the project during the Fall 2006 semester in our elementary and 
secondary education intensive field experiences. During the Spring 2007 semester, we 
implemented the project in the student teaching experience as well, guaranteeing all pre- 



service teachers had the opportunity to complete the initial project prior to student 
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teaching. As a faculty we discussed the meaning of the levels of achievement for each 

category. 

The sample included 139 pre-service teachers who were enrolled in the education 
program. Students were rated on a 3-point scale (l=Unsatisfactory, 2=Satisfactory, and 
3=Above Average) on three separate criteria (Design for Instruction and Assessment, 
Analysis of Student Learning, and Reflection on Student Learning) during their field 
experience and again during student teaching. T-tests were calculated to determine if any 
significant differences between the scores for field and student teaching existed. Table 1 
displays the mean scores for field and student teaching. As depicted in the second and 
third columns, the means tend to decline from field to student teaching. However, only 
“Reflection on Student Learning” resulted in statistically significant differences between 
field experience and student teaching. The differences in “Design for Instruction” and 
“Assessment and Analysis of Student Learning” were not significantly different (p < .05) 



Table 1: Means and t- values by criterion and experience 
(n=139) 



Criterion 


Field 

Experience 


Student 

Teaching 


t-value 


Design for Instruction 
and Assessment 


2.76 


2.67 


1.74 


Analysis of Student 
Learning 


2.68 


2.58 


1.61 


Reflection on Student 
Learning 


2.71 


2.60 


2.02 * 



As stated earlier, our goal was to have the pre-service teachers become more 
proficient in analyzing their impact on learning and modify their instruction or objectives 
to improve achievement by (1) determining the effect of instruction on all pupils’ 



learning; (2) guiding decisions about future instruction and plans to improve upon every 
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pupil’s performance; (3) communicating performance results to others; and (4) reflecting 



on their performance as teachers. 

The pre-service teachers are demonstrating competency in “Design for Instruction 
and Assessment,” as well as “Analysis of Student Learning.” Campbell and Evans (2000) 
note that it is difficult to identify pre-service teachers’ assessment decisions because each 
context is different and complex. They expound, “Because evaluation of pupil learning is 
a major component of teaching responsibilities, one must have a better understanding of 
preservice teachers’ attitudes, beliefs, behaviors, and environmental pressures as related 
to the practice and use of classroom assessment” (354). The pre-service teachers seem to 
be aware of this complexity as demonstrated in their ability to redesign instructional 
strategies as a result of formative assessments. Having learned in the college classroom 
about the different methods to help pupils achieve the objectives, the pre-service teachers 
were able to implement them in their classrooms. As seen in Emily’s “Design for 
Instruction” (pages 13 and 14), she was able to utilize various instructional strategies that 
appealed to her pupil’s interests. The pre-service teachers also demonstrated their ability 
to communicate performance results to others and reflect on their performance through 
their visual representation of pupil learning growth, as well as through their narratives, as 
explained earlier in the paper. Erom the written analysis of the project, pre-service 
teachers knew that observable objectives formulated from backward design (as 
previously discussed) were imperative to formulating successful instructional strategies. 
They also admitted to “trial and error” and modifications as they worked with students 
who continued to struggle with meeting the objectives. Emily wrote: 
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. . Jake and I completed the word sort together. He seemed to really enjoy this 

activity, and he was actively engaged throughout. Since Jake was proving his 

ability in completing this task, I had him complete the word sort for a second time 

with no assistance. He really enjoyed this, and it was as if he was in a competition 

with himself, trying to complete the word sort as quickly as possible. Lastly, I had 

Jake select one word from each of the beginning diagraphs and create a sentence 

using that word. This was originally not planned, but I was curious to see how 

Jake would respond to this task, and I wanted to make sure that he was fully 

comprehending the activity. I was pleasantly surprised! Jake’s sentences were 

creative and proved his comprehension of the activity. 

We are currently analyzing why the decline, specifically in “Reflection on Student 
Learning,” occurs in student teaching. We believe there are several reasons for this 
phenomenon. First, the demands on pre-service teachers during their student teaching 
experience are much more multidimensional and multifaceted. During field, the 
students’ major assignment is to implement the impact on student learning project. In 
student teaching, however, the students have to learn the curriculum, daily routines, and 
policies, as they are assume the larger role as teacher. It is even more difficult if the 
context they experience in their field placement varies greatly from their student teaching 
assignment because time is spent acclimating to the new context. This larger workload 
may preclude ample time to implement the project and analyze the data as those students 
field do. 

Finally, those who supervise in the field are full-time faculty teaching the 
education course in tandem with the field experience. Thus, pre-service teachers have 




Impact on Student Learning Project 24 

more opportunities to meet with the instructor for individual assistance in implementing 



and analyzing the project. Since our student teaching supervisors are not part of our full- 
time faculty, there is limited time to meet as a group. Instead, these supervisors receive 
much of their communication through memos and e-mail messages, disallowing for 
discussion that would help assure inter-rater reliability. Moon, Callahan, and Tomlinson 
(1999) purport student teachers need assistance in adjusting to the complexities of diverse 
classrooms, which often loses priority in the multi-dimensions of mentoring. Since it is 
our intent to have our pre-service teachers become more analytical and reflective in their 
practice, we plan to work more closely with our student teaching supervisors to make this 
project more of an important focus. 

Concluding Thoughts 

We are pleased with the results collected from three years of implementing this 
project. However, we realize that several issues need to be addressed. 

First, the pre-service teachers have not provided evidence of growing conceptual 
understandings among their pupils or of their pupils' ability to analyze, synthesize, and 
evaluate information. The reason may be in the way the students construct their 
project with their pupils. For example, they must collect quantifiable data from their 
pupils' assessments in order to show growth. These kinds of data are more readily 
collected by measuring increases in skill development, which does not lend itself to 
analysis, synthesis, and evaluation. This issue is being more fully discussed in the 
higher-order thinking class pre-service teachers must complete, as well as including it as 
a criterion on the rubric. Even if the pre-service teachers' reflections are culled from 
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their perceptions and interactions with the pupils— not from quantifiable data-- it will 



provide them with an awareness that may be currently lacking. 

Also, since first implementing the project, we have made changes in introducing 
the project to the pre-service teachers. In the fifth semester (the first semester of the 
Teacher Education Program), students complete a learning theory course, where they are 
introduced to stages of cognitive, social, and emotional development. They are also 
introduced to basic lesson planning. During that semester, we teach “backward design,” 
having students decide on ways to measure what is being learned. From there, students 
write objectives and formulate methods of instruction, along with appropriate activities. 
During their first field experience, they are specifically assigned to reflect on the lessons 
they are observing through the lens of backward design. In the sixth semester, we 
introduce differentiated instruction — its purpose and design. Students now include in 
their lesson plans specific ways they will address students who need more assistance, 
using scaffolding as the essential component. During this field experience, students are 
asked to reflect on the types of differentiated instruction they have observed, specifically 
detailing why and how the teacher implemented it and reflecting on the results. During 
the seventh semester, students continue to learn in their higher-order thinking class ways 
to devise more differentiated methods and to reflect critically on the results of that 
instruction. In this semester, students must demonstrate the connection of their plans to 
the developmental stage of their pupils. 

As we continue in the process of improving our students’ implementation of 
assessing individual pupil learning, we are expecting improved results in the students’ 
projects, especially in their ability to reflect critically on the results of their pupils’ 
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learning. However, Emily’s comment below is typical of the self-awareness the pre- 
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service teachers have received from the experience: 

I understand I will have to improve upon my skills as a teacher and have tons of 
room for professional growth since I am new to this and had only one pupil to 
work with for this project. I also have to remember I am not going to be able to 
get to everyone on an academic level. The most important thing for me to do is to 
try as much as I can, to be flexible in my instruction and teaching techniques, and 
be patient so that I know I have touched upon every possible angle to be an 
effective teacher who can make a difference. 

Reflective comments, such as this one, help validate the importance of the project to the 
pre-service teachers’ growth as they make the transition from student to teacher. 
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